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Abstract—Background: Several changes have been 
made to the assessment component of Saudi 
residency training programs. Among those is the 
implementation of three examinations over the 
course of the year. 

Aim: We aimed to explore the emergency 
residents’ perspective on the change in the number 
of examinations, and the impact of such changes in 
terms of time management, knowledge gain, and 
social life. 

Methods: This cross-sectional study was carried 
out from September to October 2022, using an 
electronic survey targeting emergency board 
trainees. 

Results: One hundred and nine emergency 
residents enrolled, of whom 64.2% were male. The 
majority, 45%, were from the central province. 
Junior-level residents (R1) represented 26.6% of the 
sample, while R2 (second year) comprised 18.3%, R3 
(third year) comprised 38.5%, and 16.5% were 
senior (R4) level. More than half of the participants, 
56 % (n=61), did not support the change from one to 
three examinations and believed that it had a 
negative influence on knowledge gain and clinical 
skills. The influence of the change on time 
management stands out as a negative impact, in 
addition to its impact on social life and annual leave 
arrangements. 

Conclusions: The support for three 
examinations throughout the year was low; a 
contributing factor to this may be the sudden 
changes effected by those tests on training and time 
management. A re-evaluation of testing culture and 
involving residents in decision-making might 
generate acceptance. 
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I. INTRODUCTION 


Medical and surgical trainees across the globe 
must undergo vigorous training and assessment dur- 
ing their residency years before qualifying to work 
independently. Traditionally, clinical practice and 
time spent in the clinical setting are considered the 
predominant structure for postgraduate training, with 
additional final examinations including written 
exams, oral exams, and objective structured clinical 
exams (OSCEs). Assessment tools include written 
or standardised tests, simulation-based assess- 
ments, and direct observation [1]. The movement of 
competency-based medical education (CBME) has 
given rise to multiple bodies and initiatives within 
medical education, such as the Saudi Commission 
for Health Specialties (SCFHS) in Saudi Arabia [2]; 
the General Medical Council (GMC) in the United 
Kingdom; the Accreditation Council for Graduate 
Medical Education (ACGME) in the United States; 
and the Educating Future Physicians of Ontario 
(EFPO) and CanMEDS competency initiatives in 
Canada [1]. 

Emergency medicine (EM) residency programs 
accredited by the Royal College of Physicians and 
Surgeons of Canada (RCPSC) formally im- 
plemented their CBME evaluation procedure for 
residents commencing postgraduate training in July 
2018. This assessment program consists of 28 
entrustable professional activities (EPAs), assessed 
on a five-point entrustment scale and divided into 
four stages (Transition to Discipline, Foundations of 
Discipline, Core of Discipline, and Transition to 
Practice) spread over five years of training, all of 
which were predefined centrally by the RCPSC EM 
Speciality Committee [3]. 

The Outcome Project, initiated in the United States 
by the Accreditation Council for Graduate Medical 
Education (ACGME), delineated six domains to 
determine medical competence: patient 
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care, medical knowledge, professionalism, practice- 
based learning and improvement, interpersonalcom- 
munication skills, and systems-based practice. Ac- 
cording to one survey, direct observation and global 
assessment are used, and a few programs include 
standardised patient examinations and drug pre- 
scription practice reviews in their evaluations [4]. 

In the United Kingdom, standards for postgradu- 
ate medical education and training are established by 
The General Medical Council (GMC). All specialist 
training programs describe knowledge, skills, and 
behaviours according to the General Medical 
Council’s "Good Medical Practice": knowledge, 
skills, and performance; safety and quality; commu- 
nication, partnerships, and teamwork; and sustaining 
trust. Supervised learning events (SLEs), summative 
evaluations of performance and tests, and triangu- 
lated judgment by the educational supervisor are 
used to measure competence progression. After a 
successful speciality training program, a Certificate 
of Completion of Training (CCT) is awarded and 
permits entry into the specialist list [5]. 

In the Netherlands, competence is used to evaluate 
postgraduate medical education. All new residents 
work with the director of the training program to 
create an "individual training plan" based on abilities 
gained before speciality training, such as during 
previous employment as a resident-not-in- training. 
During speciality training, residents create a 
portfolio documenting their progress across all skill 
areas, and this portfolio is the foundation for each 
resident’s progress reviews [5]. 

Such international training programs lack agree- 
ment on a standardised method of assessment. The 
variation in assessment, although agreed on the need 
for impartial assessment using different means and 
methods, still renders the number of examinations 
controversial. In Saudi Arabia, the assessment model 
includes a yearly written examination, in addition to 
OSCE and evaluation per rotation. 

In 2020, a change was made to the number of 
written examinations in the emergency board 
training — from one per year to three, and a logbook 
became a requirement for mini-clinical evaluation 
exercises (mini-CEX) and direct observation of pro- 
cedural skills (DOPS). The latter two requirements 
were based on evidence of their valuable utility in 
medical education [6-8], and an argument could 
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be made for their informal application before the 
effective change date. However, while the change in 
the number of assessments was intended to engage 
residents more with the educational material, its 
efficiency from the residents’ perspective still needs 
to be evaluated. This study, therefore, aimed to assess 
the perspective of EM residents on the recent change 
in the number of examinations, explore its impact on 
their knowledge gain, time management, and 
reported stress and anxiety, and compare those 
factors between different training provinces. 


Il. METHODS 


This is a cross-sectional study carried out from 
September 2022 to October 2022. The survey was 
created by the authors and distributed using an online 
platform. A convenient sample was sought targeting 
emergency medicine resident trainees in Saudi 
Arabia. Sampling was done through contacting 
residents’ representatives from various ge- 
ographical regions and asking them to distribute the 
survey within their centres. Trainees’ level donated 
by R1-R4 represents the level of training from junior 
to senior level. We used Fisher’s Exact Test to 
explore any significant association that might 
influence our research inquiry. We considered a level 
of significance to be less than 0.05. We used SPSS 
version 25.0 for statistical analysis. 

The study was approved by the IRB committee, 
with log number 22-407. 


Il. RESULTS 


Demographics: 

One hundred and nine emergency residents re- 
sponded to the survey. The participants’ demograph- 
ics are illustrated in Table 1. 


Attitude toward the change from one exam to three 
exams: 

More than half of the participants, 56 % (n=61), 
supported having one exam instead of three, whereas 
14.7% (n=16) were neutral in supporting or opposing 
three exams. 

There was a significant difference between 
provinces regarding the attitude toward having three 
exams; Fisher exact=24.967, p <0.005. Trainees 
from the Central Province were more supportive of 
having three exams, at 78.1%, than trainees from 
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TABLE 1. Demographics of the participants 


Variables Percentage (n) 
Gender 
Male 64.2 (70) 
Female 35.8 (39) 
Province of training 
Central Province 45 (49) 
Southern Province 20.2 (22) 
Eastern Province 10.1 (11) 
Western Province 24.8 (27) 
Residency level 
RI 26.6 (29) 
R2 18.3 (20) 
R3 38.5 (42) 
R4 16.5 (18) 


the Southern, Eastern, and Western provinces: 6.3%, 
9.4%, and 6.3%, respectively (Figure 1). The atti- 
tude toward the three exams did not differ between 
junior and senior training levels; p=0.467. 


The influence of three exams on knowledge gain 
and clinical skills: 

Less than half of the participants reported a neg- 
ative influence on knowledge gain, 43.1% (n=47), 
compared to 29.4% (n=32) who were neutral in their 
answer. Likewise, approximately two-thirds reported 
a negative influence on the gain of clinical skills 
62.2% (n=46), compared to one-third who were 
neutral 33% (n=36). 

The effect of three exams on knowledge gain was 
not statistically different between provinces; 
p=0.127. Compared with other provinces, we noted 
that trainees from the Central Province disagreed on 
the negative influence of three exams on the gain of 
clinical skills; Fisher’s Exact Test=22.648; p=0.001. 


The influence of three exams on time management: 

Most of the participants, 78% (n=85), reported 
having difficulties with time management. Likewise, 
83.5% (n=91) reported a negative influence on their 
social life and an adverse impact on their annual 
leave arrangements; 88.1% (n=96). 

There was a significant difference between 
provinces regarding the effect of three exams on time 
management, Fisher’s exact test =19.82 (p = 0.001)), 
as illustrated in Figure 2. 


The stress and anxiety from having three exams: 


Although trainees reported stress and anxiety, and a 
negative impact on social life, nonetheless it did 
not differ between different trainees’ level or 
between provinces; p>0.05. 


Gender difference in relation to the new changes: 
Although males represented two thirds of our 
sample, 64.2% (n= 70), the participants’ gender did 
not have a statistically significant influence on their 
support for or objection to having three exams. It 
also had no impact on knowledge gain, clinical 
knowledge, time management, or social life; p>0.05. 


IV. DISCUSSION 


This data demonstrates a low rate of acceptance of 
the move from one test to three, with no gender or 
trainee level differences. The low acceptability is 
most likely owing to the abrupt changes imposed by 
these assessments on training or time management. 
Interestingly, trainees reported a negative influence 
on their knowledge gain and clinical skills, al- 
though this is counterintuitive to what we know from 
the learning theories in which consolidation and 
retrieval and development of long-term learning are 
enhanced by frequent examinations [9]. One 
explanation might be the time constraints associated 
with reading about every case encountered after the 
end of a shift, with also having to focus more on the 
topics of an exam. Perhaps the testing culture, in 
which the test result is viewed as more important, 
influences people’s perceptions of the value of three 
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exams. Another explanation might be that the idea 
of these exams being opportunities to learn, rather 
than tests per se, has not been adequately conveyed. 
Nonetheless, a gradual implementation of any 
changes, and involving the residents in those 
changes, fosters commitment and greater acceptance 
[10]. The true impact of the changes to the assess- 
ment can be measured objectively using the median 
knowledge curve before and after the change. This 
can be obtained by comparing test scores from 
before and after the change. 

Time management appears to have a central role 
in the residents’ attitude toward the change. A 
focus on time management should be part of any 
quality improvement project, particularly when it 
influences a busy training curriculum like that of 
emergency medicine. The residents might, for 
example, be asked to agree upon scheduling the 
exams at the beginning of the academic year, which 
might accommodate the time constraints and the 
effect on annual leave and social life. Furthermore, 
productivity and well-being are enhanced after time 
management workshops are conducted for residents 
[11]. 

Although less than one third supported the new 
changes, support was higher in the Central region 
than in the other provinces. The Central Province 
also refutes any negative influence on time man- 
agement or the gain of clinical skills. This raises 
questions regarding the implementation of the three 
exams or the similarity in teaching style between 
different provinces. More research into this notion is 
justified. 

The convenient sample used for this analysis rep- 
resents a limitation thereof. We believe more inquiry 
is needed into how the changes to the assessment 
should occur, including seeking the perspectives of 
both educators and trainees. 

Overall, support was low for having three exams 
throughout the year. Further assessment of the 
benefit of three exams should be undertaken, and it 
should be noted that the involvement of the residents 
in the education and assessment process is 
paramount to fostering commitment. 
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